3D sensors, in particular lidar, radar and similar systems, increasingly become the main source of data used for both manned and unmanned vehicles operating in demanding environments. They are providing data e.g. for obstacle warning systems and for landing aid systems. For operational systems the environment is not a clinically well-defined laboratory but data processing has to cope with all aspects of a natural or man-made environment in real-time. When using 3D sensors for the active support of aerial vehicles or autonomous ground vehicles, processing the measured 3D data becomes a central task. The amount of data generated by 3D sensors can be tremendous. Therefore, a fast, intelligent and efficient way of data reduction has to be found. This invention is concerned with data selection and reduction at the very first steps in the evaluation of the sensor data (conducted in an analogue pulse analyzer as a subsystem of the 3D sensor).
Particularly for optical sensors used for obstacle warning and as landing aid in degraded vision environment (DVE), requirements for detection are diverse, sometimes competitive, or even contradictory. Primarily, the sensor can work as an obstacle warning system where it is very sensitive to small signals provided by the echo of thin wires in the range from tens of meter to more than 1000 m. For doing this, radiation can pass through air that might be obscured by fog, clouds, dust etc. The echoes originating from these obscurants must not damage the sensor and ideally should be ignored by the 3D sensor. In addition, the sensor can detect ground in the same distance range mentioned before. As the back reflected intensity decreases with 1/r2 or 1/r3 for bulk targets and for wire targets, respectively, the signal receiver has to cope with signal intensities that cover several orders of magnitude.
There are currently two approaches in common 3D measurement systems based on photon runtime measurement, i.e. time of flight counters. The first known art is to record the complete returned intensity sequence, digitize and analyze it digitally in real time. This evaluation requires enormous computing power due to the huge amount of data points on the one hand and the enormous signal dynamics on the other hand ([MUR13, ULL05a, ULL05b]).
Another known approach is to extract discrete points of time by applying a trigger condition on the signal train. The invention as described in the following will apply this particular approach.
A schematic representation of a system used for carrying out such a method is depicted in FIG. 1. An emitter 11 sends pulses of radiation 15 (e.g. laser or radar pulses). Part of the intensity is scattered back or reflected back by an object 16 (e.g. ground, wires, obscurants) and is received by a receiver 12. The signal intensity train or signal amplitude train (short: signal train) is investigated in a pulse analyzer 13 producing one or a series of measurement distances which are prepared for further processing 14.
FIG. 2 shows an example of a signal train 21 as received by a receiver 12.
The signal train includes several signal peaks evoked by scattering of the emitted pulse by objects in different distances. The signal strength of the returning pulses is compared to a trigger threshold 41 (hereinafter also called trigger level). A peak exceeding the trigger threshold 41 is called an echo pulse (or echo for short). The time of such event (hereinafter called trigger time) is stored and such event is called a trigger event 23. Trigger time of an echo and the distance of the object that gave rise to this particular echo are connected via the velocity of the electromagnetic radiation emitted by the 3D sensor which can be taken as constant. Hence, trigger time and object distance are equivalent and can both be used for the horizontal axis of the diagram of FIG. 2 (this applies also to the diagrams of FIG. 3 to 6). As shown in FIG. 2, the trigger level 41 may be time-dependent, i.e. decreasing with time, to compensate for signal decrease with distance. As a result, undesired events, in particular caused by fog and snow, are suppressed. Those signals are often much smaller than signals of hard targets. An example of such time dependent trigger condition is disclosed in DE 10 2012 021 831 A1. There, a step function is applied. For short distances (i.e. short measurement times), a higher trigger level is applied than for longer distances (i.e longer measurement times).
In the example of FIG. 2, a so-called rising edge trigger is used, i.e. the trigger time is defined by the time the signal strength reaches the trigger level on the rising edge of the echo pulse. There are numerous other well-known trigger methods in order to determine an echo pulse and to derive a trigger time there from. Simple methods are comparators and Schmitt triggers where the signal train is compared to trigger thresholds and the time of the rising or the falling edge of the threshold crossing is recorded. More sophisticated methods are pulse width triggers and constant fraction discriminators [GED68]. The trigger conditions typically contain one or more trigger parameter, like threshold, rise time, etc. The variety of conditions can be extended by combining or cascading trigger methods and parameters. In this manner, trigger conditions can be tailored for expected patterns of the signal train.
Additional measures are known in order to dismiss undesired echoes and thus limiting the number of echoes to be processed. They may stem from dust, fog, etc. in the vicinity of the sensor. A frequently used technique to avoid such undesired trigger events is to use the so-called range gating. Echoes arriving from distances smaller than a minimum distance (hereinafter: the near range distance) are suppressed thus creating a blind zone in the direct vicinity of the 3D sensor.
The echoes of the signal train 21 which are received from outside said blind zone (and originating in the same signal pulse emitted by the 3D sensor) are intuitively labeled 1st echo, 2nd echo, 3rd echo, last echo, etc. according to their rank in the chronological order in which the echo pulses are received. It has been reported that up to 16 echo pulses of these sequence of pulses are being processed [TRI13]. Here, too, computing power limits the number of echo pulses to be processed. Interface bandwidth and the need for real-time processing are highly limiting factors, especially in airborne applications. When the correlation between different echoes is investigated, the required processing power increases tremendously with the number of returns processed. As a consequence, frequently only two echoes are exploited [SEI08]. They are typically an acceptable compromise between the need of information of more than one echo and the available computing performance.
A known two-echo approach [STE15] that exploits first and second echo uses a so-called delta range gate: One defines a so-called blind zone immediately behind (as seen from the 3D sensor) the object that caused the first echo. Any trigger event caused by an echo from objects within the blind zone (in the time domain this corresponds to a certain time slot after arrival of the first echo), will be ignored. Hence, the second trigger event can only be caused by an echo arriving from distances behind the blind zone.
The less echoes that are processed, the more crucial it is to choose those echoes for further processing that contain the most useful information. The selected echoes shall produce measurement distances of real objects instead of artifacts. In the context of obstacle warning under normal weather conditions, processing of the 1st echo is inevitable. In most cases it contains the echo of the actual obstacles. Commonly, a 1st/2nd echo scheme or a 1st/last echo scheme is used. The second echo seems to be the best choice for obstacle detection in most DVE situations. Here, the first echo might be an echo from an obscurant e.g. dust, rain, fog or snow and the second echo might stem from an actual obstacle. Ground detection is another key function. It is the main task during lift off, approach and landing and, in addition, it is a significant side functionality in obstacle warning. For ground detection, the last echo is to be preferred as ground will always cause the very last echo.
Irrespective of the number of echoes processed, conventional 3D sensors use only one trigger condition that is used for the determination of all echoes processed.
As the intensity scattered by the objects that give rise to an echo is highly dependent of their distance to the sensor, it is the object of the invention to provide a method for processing echo pulses which provides an improved suppression of undesired trigger events.
According to the present invention different trigger conditions, like trigger thresholds, are used for the determination of the pre-defined distinct echo pulses that are to be exploited. These trigger conditions are each used simultaneously. The evaluation can be performed in separate process channels.
By doing so, the different trigger conditions can be adapted to typical echo distances, expected strength and rank in the chronological order of the individual echoes to be processed. For example, a trigger threshold for determining the first echo will generally be higher than a trigger threshold for the determination of a later echo. In a preferred embodiment, the conditions or parameters in both channels can be time dependent. Another advantageous option is to apply a time dependence on the trigger condition of at least one channel, but not of all channels.
The application of different trigger conditions is done in separate process channels so that, for each trigger condition applied, an independent series of trigger times/echo distances is recorded. For example, reception and recording of trigger times of the echo pulses is done in parallel in separate pulse analyzers applying different trigger conditions to the sequence of received signal pulses.
It is an important advantage that the described evaluation of the distinctive echo pulses by using individual trigger conditions, e.g. trigger levels, allows independent optimization with respect to the intended purpose of the 3D data. One important special case is ground detection under brownout conditions. Here, the intensity reflected from the cloud of dust may be strong and the transmitted intensity may be highly attenuated even in short distances. Therefore, an increase of trigger level is only advantageous with respect to the suppression of undesired trigger events for a first echo and a second echo but not for a last echo.
Furthermore, the use of independent trigger conditions allows qualitative conclusions about echo signal strength or shape. For instance, in a process applying two different trigger levels (relating to first echo/second echo), the fact that an echo has fulfilled both trigger conditions may provide useful additional information. In cases where there is only a second echo but no first echo or in cases where the trigger time of the second echo is earlier compared to the trigger time of the first echo, the information is already inherently included in the distance information. In the important case when the trigger level for the second echo is lower than that for the first echo the second echo must be a weak echo.
The method according to the invention is applicable for all types of 3D measuring devices with time of flight measurements and subsequent pulse analysis resulting in discrete values for echo distances.
The method according to the invention satisfies the requirements for a wide range of 3D sensor applications and operational situations, especially for optical 3D scanners. It can be completely implemented in the analogue pulse analyzer of the 3D sensor. The described logical operations performed on the signal train are easy to implement in hardware or firmware.
It is a further advantage of the method according to the invention that the filtering process according to the invention is performed at an early stage of the complete processing chain where it is most effective. The invention reduces the information of each single 3D measurement to discrete distance values that contain the desired information and that are affected by artifacts as little as possible.
The 3D sensor data (i.e. the distinct echo pulses) generated according to the present invention can be further processed according to well-known techniques. In particular the 3D data can be fused with navigational data in order to provide geo-referenced 3D data of the surroundings. Filtering operations can be conducted to remove the adverse impact of sun, cloud, dust, drop-in etc. on the 3D data. Further, a classification of objects can be performed and warnings and visualization outputs can be generated and brought to the attention of the human operator of the vehicle.